Skip to content

feat(replay): Add beforeStoreFrame callback for snapshot testing#5386

Open
runningcode wants to merge 5 commits into
mainfrom
no/java-504-replay-before-store-frame-callback
Open

feat(replay): Add beforeStoreFrame callback for snapshot testing#5386
runningcode wants to merge 5 commits into
mainfrom
no/java-504-replay-before-store-frame-callback

Conversation

@runningcode
Copy link
Copy Markdown
Contributor

Summary

  • Add an experimental BeforeStoreFrameCallback to SentryReplayOptions that fires right before a replay frame is stored to disk, receiving the masked bitmap, timestamp, and screen name via Hint
  • Add a Kotlin extension SentryReplayOptions.beforeStoreFrame { bitmap, ts, screen -> } for ergonomic usage
  • Add ReplaySnapshotTest UI integration test that captures masked replay frames via TestStorage on Sauce Labs
  • Enable useTestStorageService and *.png artifact collection in the Sauce Labs config, with sentry-cli build snapshots upload in CI

Relates to JAVA-504

🤖 Generated with Claude Code

Add an experimental callback that fires right before a replay frame is
stored to disk. The callback receives the masked bitmap (via Hint),
timestamp, and current screen name. This enables snapshot testing of
replay masking without needing to decode stored video segments.

Includes a Kotlin extension for ergonomic usage:
  options.sessionReplay.beforeStoreFrame { bitmap, ts, screen -> ... }

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 8, 2026

JAVA-504

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

Messages
📖 Do not forget to update Sentry-docs with your feature once the pull request gets approved.

Generated by 🚫 dangerJS against 983a3f3

@sentry
Copy link
Copy Markdown

sentry Bot commented May 8, 2026

📲 Install Builds

Android

🔗 App Name App ID Version Configuration
SDK Size io.sentry.tests.size 8.41.0 (1) release

⚙️ sentry-android Build Distribution Settings

@runningcode runningcode force-pushed the no/java-504-replay-before-store-frame-callback branch 2 times, most recently from 5b10cdd to e7452f7 Compare May 8, 2026 08:36
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 311.04 ms 361.61 ms 50.57 ms
Size 0 B 0 B 0 B

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
674d437 355.28 ms 504.18 ms 148.90 ms
2195398 321.31 ms 391.66 ms 70.35 ms
62b579c 312.88 ms 361.57 ms 48.70 ms
b77456b 393.26 ms 441.10 ms 47.84 ms
694d587 312.37 ms 402.77 ms 90.41 ms
6b019b7 319.84 ms 333.15 ms 13.31 ms
6edfca2 305.52 ms 432.78 ms 127.26 ms
b03edbb 314.90 ms 350.22 ms 35.33 ms
d15471f 294.13 ms 399.49 ms 105.36 ms
fcec2f2 314.96 ms 373.66 ms 58.70 ms

App size

Revision Plain With Sentry Diff
674d437 1.58 MiB 2.10 MiB 530.94 KiB
2195398 0 B 0 B 0 B
62b579c 0 B 0 B 0 B
b77456b 1.58 MiB 2.12 MiB 548.11 KiB
694d587 1.58 MiB 2.19 MiB 620.06 KiB
6b019b7 0 B 0 B 0 B
6edfca2 1.58 MiB 2.13 MiB 559.07 KiB
b03edbb 1.58 MiB 2.13 MiB 557.32 KiB
d15471f 1.58 MiB 2.13 MiB 559.54 KiB
fcec2f2 1.58 MiB 2.12 MiB 551.50 KiB

Previous results on branch: no/java-504-replay-before-store-frame-callback

Startup times

Revision Plain With Sentry Diff
cd28dc9 316.30 ms 354.64 ms 38.34 ms
62bcea4 359.22 ms 426.90 ms 67.67 ms
1cddad0 342.73 ms 424.14 ms 81.41 ms

App size

Revision Plain With Sentry Diff
cd28dc9 0 B 0 B 0 B
62bcea4 0 B 0 B 0 B
1cddad0 0 B 0 B 0 B

…(JAVA-504)

Add ReplaySnapshotTest that uses the beforeStoreFrame callback to
capture masked replay frames during a Compose UI test. Frames are
written to the Downloads/sauce_labs_custom_screenshots/ directory,
which is the standard path Sauce Labs collects screenshots from.

CI changes:
- Add *.png to Sauce Labs artifact match patterns
- Upload collected replay snapshots via sentry-cli build snapshots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@runningcode runningcode force-pushed the no/java-504-replay-before-store-frame-callback branch from e7452f7 to 2aff4b0 Compare May 8, 2026 08:46
…VA-504)

The Kotlin extension `beforeStoreFrame` comes from `sentry-android-replay`
which may not resolve in the UI test module. Use the Java callback API
directly instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@runningcode runningcode marked this pull request as ready for review May 8, 2026 12:18
…VA-504)

GH Actions emulators don't support screenshot capture for replay,
so the ReplaySnapshotTest needs the same assumeThat guard used by
ReplayTest. Also adds a changelog entry for the beforeStoreFrame
callback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comment on lines +314 to +323
val callback = options.sessionReplay.beforeStoreFrame
if (callback != null) {
try {
val hint = Hint()
hint.set(TypeCheckHint.REPLAY_FRAME_BITMAP, bitmap)
callback.execute(hint, frameTimeStamp, screen)
} catch (e: Throwable) {
options.logger.log(ERROR, "Error in beforeStoreFrame callback", e)
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: A race condition exists where the shared screenshot bitmap can be overwritten by the next frame's PixelCopy operation while the beforeStoreFrame callback is still processing it.
Severity: HIGH

Suggested Fix

Provide a copy of the screenshot bitmap to the beforeStoreFrame callback instead of the shared instance. This can be achieved by creating a new bitmap using bitmap.copy(bitmap.config, true) before invoking the callback. This ensures the callback operates on an isolated version of the frame data, preventing concurrent modification issues.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location:
sentry-android-replay/src/main/java/io/sentry/android/replay/ReplayIntegration.kt#L314-L323

Potential issue: The `beforeStoreFrame` callback is passed a shared, mutable
`screenshot` bitmap that is reused across all frames. This callback executes on a
background thread, `replayExecutor`. Concurrently, the main thread schedules the next
frame capture, which uses `PixelCopy` to write to the same shared bitmap. There is no
synchronization between the `PixelCopy` write operation and the `beforeStoreFrame`
callback reading the bitmap. If the callback performs I/O, such as `bitmap.compress()`,
the bitmap's data can be overwritten by the next frame's data mid-read, leading to
corrupted images.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, but that's already tracked here: #5340

@Before
fun setup() {
// GH Actions emulators don't support capturing screenshots for replay
@Suppress("KotlinConstantConditions")
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from ReplayTest but why are we even running these on emulators in gh actions?

Copy link
Copy Markdown
Member

@markushi markushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Let's discuss first if the API should be experimental for internal, but other than that no blockers.

Comment thread sentry/src/main/java/io/sentry/SentryReplayOptions.java Outdated
* intercepting frames for testing (e.g., screenshot comparison tests) or custom processing. The
* callback receives the frame after masking has been applied.
*
* <p>The frame bitmap is passed via a {@link Hint} using the key {@link
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

mkdirs()
}
val frameReceived = CountDownLatch(1)
val capturedScreens = CopyOnWriteArrayList<String>()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not that it matters too much for tests, but that could be a set as well, right?

Suggested change
val capturedScreens = CopyOnWriteArrayList<String>()
val capturedScreens = mutableSetOf<String>()

Comment on lines +314 to +323
val callback = options.sessionReplay.beforeStoreFrame
if (callback != null) {
try {
val hint = Hint()
hint.set(TypeCheckHint.REPLAY_FRAME_BITMAP, bitmap)
callback.execute(hint, frameTimeStamp, screen)
} catch (e: Throwable) {
options.logger.log(ERROR, "Error in beforeStoreFrame callback", e)
}
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, but that's already tracked here: #5340

Co-authored-by: Markus Hintersteiner <markus.hintersteiner@sentry.io>
Copy link
Copy Markdown
Member

@0xadam-brown 0xadam-brown left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this @runningcode. A few comments.

Big picture, did we think about defining a new ReplayScreenshotObserver interface inside the replay module (akin to ScreenshotRecorderCallback) rather than routing through the (universally available) SentryReplayOptions?

An interface in the replay module would let us avoid the Hint indirection, and it'd avoid the tension of putting @ApiStatus.Internal on an *Options member.

Thoughts?

* <p>The callback runs on a background thread (replay executor). Do not recycle the bitmap — it
* may be reused by the replay system.
*/
@ApiStatus.Experimental
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

h: Unless someone has asked for this callback, my vote would be for Internal rather than Experimental to avoid a footgun.

Eg, looks like we're hoping for read-only use cases, but we don't protect against writes, meaning users could modify the screens we save for session replay. And we lock on the bitmap after handing it off, meaning deadlock becomes a possibility (at least analytically). We could hand off a copy but I imagine we'd still want to recycle it after the callback completes to avoid memory issues, and that means warning the user accordingly. Etc.

Thoughts?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...looks like I was late to this game and we've updated to internal - good deal. (We should update the other sites to match 👍 )

* may be reused by the replay system.
*/
@ApiStatus.Experimental
public interface BeforeStoreFrameCallback {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: Thoughts about keeping the nomenclature in line with our "screenshot recorded" language?

Eg, renaming BeforeStoreScreenshotCallback (if we want to guarantee the callback is fired immediately before writing to disk), or ScreenshotRecordedCallback (if we want to keep our options open about exactly when we invoke it) .

* TypeCheckHint#REPLAY_FRAME_BITMAP}. On Android, retrieve it with: {@code hint.getAs(
* TypeCheckHint.REPLAY_FRAME_BITMAP, Bitmap.class)}.
*
* <p>The callback runs on a background thread (replay executor). Do not recycle the bitmap — it
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: We could beef this up to protect against misuse if we think it'd help:

 * <p>**Bitmap lifecycle:** The bitmap is owned by the replay system. It should be deemed 
 * read-only and valid solely for the duration of the callback. Do not write to it or recycle it. Do
 * not synchronize on it, store a reference to it, or access it after this method returns – copy
 * the pixel data (e.g., compress to a file) within this method if you need it later.
 *
 * <p>The callback runs on a background thread (the replay executor).

...or whatever makes sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants